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ABSTRACT 

Student retention continues to be an important 
concern among college and university officials; many programs have 
been implemented at postsecondary institutions to improve student 
retention, especially that of minority students. This paper examines 
the theory and practice of program evaluation in retention-focused 
programs. Program evaluations typically assess four different 
criteria for program participants: (1) reaction; (2) learning; (3) 
behavior changes; and (4) results. Such evaluations try to determine 
a program's worth, but a program's effectiveness can be difficult to 
validate. Even if a positive outcome is realized, there is no way of 
establishing which of the interventions actually resulted in the 
improved retention. A quasi-experimental approach is recommended for 
assessing retention programs where participants are not randomly 
selected, but do undergo pretest and posttest observations and are 
compared to control groups. An example is presented of such an 
evaluation by evaluating a 5-week summer bridge program designed to 
help minority students adjust to college life. The survey, which was 
developed in six stages, did show that the program helped students 
prepare for and adjust to college, but the survey did not document 
the program as an effective retention intervention. A number of 
ongoing evaluations are recommended for measuring retention 
effect iveness • (RJM) 
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Summary 

The retention of students is an important and growing concern among college and 
university officials. To address this concern, numerous programs have been implemented which 
aim to improve the retention of students, especially minority students, at our institutions of higher 
learning. The main purpose of this paper is to discuss one critical component of retention 
program design and implementation —program evaluation. Evaluation is critical in order to 
answer the question of whether the program was effective. Toward this end, we discuss the 
rationale for evaluation, including types of criteria that are typically used, and methodological 
issues of importance, including the threats to establishing whether a program was actually 
effective. As an illustrative example, an evaluation of a bridge program undertaken at the Georgia 
Institute of Technology is discussed. A challenge is issued to those involved in retention program 
efforts to include the critical component of program evaluation in current and future student 
retention endeavors. 
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Introduction 

The decision-makers of post-secondary institutions, facing restrictions on financial 
resources, a decline in the traditional-age freshman pool, and changes in the enrolled 
undergraduate composition, have renewed their emphasis on retaining the alarming number of 
potential dropouts. From at-ri. k identification procedures to planned support innovations, 
university administrators yearn for more productive and satisfying educational experiences among 
students aspiring to earn the bachelor's degree. 

Background on the Retention Emphasis 

During the 1960's, numerous writings about the college retention problem became evident. 
Presidents Kennedy and Johnson frequently spoke of the changing role of higher education in the 
pursuit of social improvement. Research and demonstration funding became available for 
institutional research on student behaviors as seminal legislation was passed by Congress. 
Financial aid was bolstered; enrollment was enhanced. The Brookings Institute, for example, 
sponsored a thorough review of personality factors in college success (Lavin, 1965), resulting in 
designated characteristics of the "high achiever." The American Council of Education launched 
studies 4 > identify the most promising predictors of academic progress and degree realization 
(Astin & Panos, 1969). Models of attrition gained prominence in the 1970 ! s (e.g., Tinto, 1975). 

Almost three decades after The Higher Education Act of 1965, we are experiencing 
renewed calls for widespread reform of undergraduate education. Boyer ! s book entitled College 
(1987), sponsored by The Carnegie Foundation for the Advancement of Teaching, is one 
example. Earlier, a 1984 National Institute of Education report focused on the provision of more 
student-orieiited resources in the first year of campus life (i.e., emphasis on "front-loading" to 
promote intense intellectual interaction between students and instructors) 
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Rationale for Evaluation 

Without some kind of systematic evaluation we will not know if the program objective 
was accomplished. We live in an era of accountability , making evaluative information vitally 
important to program staff and sponsors. In most instances, we are expected to "prove 11 that what 
we do is valuable. Toward that end, evaluation planning and implementation represents an 
applied, largely (and unfortunately) atheoretic, multi-disciplinary activity directed toward the 
provision of useful, policy-relevant information (Wortman, 1983). 

It can be instructive for those directing educational interventions to highlight four criteria 
often used in the evaluation of training programs (Kirkpatrick, 1976): 

1) Reaction criteria. These criteria concern participant impressions and feelings. One 
asks questions such as u was the program useful?", or "did the program add to your 
knowledge?" 

2) Learning criteria. Rather than simply seeking participant reaction, these criteria 
concern the learning gained. Questions deal with how much was learned and the kind of 
knowledge and/or skill learned from a particular program or intervention. 

3) Behavioral criteria. Criteria with a behavioral emphasis focus on actual changes in 
participant performance over time, rather than just reaction or learning. The question is 
one of whether performance has improved as a result of the program or intervention and 
whether it has maintained over time. 

4) Results criteria. Criteria of this type are thought to be of the most value. Results 
criteria compare the benefits of the program with the associated costs. Results criteria can 
be differentiated from behavioral criteria because the former consider costs, while the 
behavioral indices do not. 

The latter two indices of program effectiveness (i.e., the behavioral and results data) are 
viewed as external to the designated project or innovation. 
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We are urged by the available evaluation guidelines to have a broad content and temporal 
perspective as we design projects and reach conclusions relating to the outcome variables we 
incorporate into our retention-oriented projects. Establishing valid criterion components is of 
course critical in the evaluation process. As emphasized in the methodological discussion to 
follow, our ideal is to obtain evidence that the intervention made a difference (i.e., met the stated 
objectives.) In many instances, project results within the SUCCEED context will have policy 
implications. 

Methodological Issues in Retention-Focused Programs 

Systematic evaluation should address the policy question of a program's worth. Was the 
program effective in the prescribed time period? • What is meant by effectiveness entails the proper 
choice of the criterion (or dependent), measure, what we are trying to improve or increase by 
implementing a particular intervention. In student retention research, the criterion has 
traditionally been the retention rate (i.e., how many students stayed enrolled in the institution). 
Another commonly used criterion in the literature is an index of student achievement, such as 
degree attainment or years to a degree. The issue of determining the proper criterion (or criteria), 
while of extreme importance, is beyond the scope of this paper. A major point is that no matter 
what criterion is chosen as the target of an intervention, it is necessary to be able to ascertain 
whether the new program was effective in causing a change in the criterion measure. According 
to Levin and Levin (1993), this evaluation principle requires controlled experimentation and 
represents the only form of scientific inquiry that will enable the evaluation process to attribute 
program outcomes directly and unambiguously to retention program characteristics. In other 
words, it is essential to determine which components of an intervention are effective and should 
be included in future retention efforts. 

The Search for Causation . Establishing cause should be the primary objective in retention 
research. The goal is to assess program effectiveness. We want to be able to say that the 
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program (intervention or treatment) was the cause of the change in the retention rate, or the cause 
of the increase in degree attainment. The core of the theory of causation underlying the design of 
experiments is to estimate the effects of a treatment contrast (Cook, et al, 1990). Appropriate to 
our current retention emphasis, Campbell and Stanley (1966) made the point that experimentation 
is "the only means for settling disputes regarding educational practice, as the only way of verifying 
educational improvements, and as the only way of establishing a cumulative tradition in which 
improvements can be introduced without the danger of a faddish discard of old wisdom in favor 
of inferior novelties." 

Several disclaimers are in order. This paper does not claim that all research needs to be 
experimental in nature to be beneficial. Other types of research efforts, such as descriptions of 
specific programs and overviews of current trends are of great value in increasing our knowledge 
of the student retention issue. Also, the present cautions do not attempt to place blame on 
retention researchers. These issues are of relevance to all participants in the educational arena. 
All of us must admit that we have fallen short of the ideal on many occasions. Due to practical 
constraints (time, financial, manpower, etc.), designing the "perfect" evaluation procedure is often 
an impossibility in the "real" world. However, we believe that paying more attention to 
methodological issues will result in a better understanding of what can be done realistically to 
improve student retention and persistence through the baccalaureate. In this way, current and 
future students should benefit from our concerted efforts. 

Lessons From the Retention Literature . As reported in the meta-analysis by Kulik, Kulik, 
and Shwalb (1983), out of 500 retention program research studies reported in the literature, only 
60 (12%) were of acceptable methodological quality. While one might (hopefully) assume that 
the last eleven years has brought methodological improvements, the retention program literature 
is still rife with studies of less than ideal research design and the improper use of statistical tests. 

At this point it is important to define some relevant terms. The term "experiment" 
typically carries with it the connotation of laboratory experiments. Obviously, in retention 



ERLC 



7 



Evaluation Considerations 

7 

• « 

research we are generally not concerned with this type of experimentation. We are primarily 
focused on field experimentation. As discussed by Cook, Campbell, and Peracchio (1990), by 
f:e!dwe mean any setting that participants do not perceive as having been set up for the primary 
purpose of conducting research. By experiment, we mean any experimenter-controlled or 
naturally occurring event with rapid onset (a treatment or program) whose possible consequences 
are to be empirically assessed. Experiments are traditionally divided into two types. Randomized 
experiments assign participants to treatments at random, while quasi experiments primarily 
depend on self-selection or administrative decisions to determine who is to be exposed to a 
treatment. Quasi experiments are typically the most relevant for retention program evaluation 
efforts, being much easier to implement than randomized experiments in field settings where 
causal conclusions are needed. However, as further discussed by these authors, a critical point is 
that quasi experiments can have all the major structural features of experiments, including pretest 
and posttest observations and comparison groups. 

Threats to Validity 

One must be aware of various factors than can influence our ability to answer the question 
of effectiveness in the affirmative. While in practice it may never be possible to rule out all of 
these threats, effort needs to be directed toward the minimization of these threats, and to make 
informed decisions of how to design a retention intervention so that one can be as confident as 
possible that the program, and not some extraneous factor, did indeed influence retention. In this 
way, others can use this information to benefit their retention efforts. 

It can be instructive to us to summarize these threats to validity in four general categories: 
(1) threats to statistical conclusion validity, (2) threats to construct validity, (3) threats to internal 
validity, and (4) threats to external validity. While all four types of validity will be defined, the 
focus of this article will be on the later two —threats to internal and external validity. For more in- 
depth information on these threats, references such as Cook, et al. (1990) are helpful. Following 
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an overview of these threats, we will discuss an evaluation of one retention effort undertaken at 
the Georgia Institute of Technology and will offer practical recommendations that will be useful in 
improving retention evaluation efforts. 

Threats to Statistical Conclusion Validity . Several conditions have to be met to conclude 
that two variables are causally related and that the direction of causation is from A (a retention 
intervention) to B (i.e., retention rate, etc.) First, a cause must precede the effect in time. 
Second, the treatment (A) and effect (B) must covary. Statistical tests are typically used to decide 
whether the covariation is "real." Again, covariation is a requirement for cause and statistical 
tests are usually used to make such judgments. Factors which can lead to false conclusions about 
covariation are called threats to statistical validity. 

Threats to Construct Validity . Another concept deals with whether the operations used to 
ascertain whether A causes B actually represent the theoretical constructs that the researcher 
claims they do. Threats to construct validity are therefore threats to the correct labeling of the 
cause and effect operations in abstract terms that come from common language and formal 
theory. 

Threats to Internal Validity . Another necessary condition for causal inference is that 
there are no plausible alternative explanations of B other than A. In other words, there are no 
other explanations for the increase in student retention other than the intervention program that 
was implemented. 

For example, it commonly occurs that at-risk students undergo a variety of retention 
interventions concurrently. The same students might be housed in special buildings, provided 
with tutoring resources, enrolled in college preparation courses, etc. Even if a positive outcome 
is realized, there is no way of telling which of the interventions actually resulted in the improved 
retention. Another possibility is that these students would have attained the same retention rate 
without special interventions of any kind. Threats of this type are called threats to internal 
validity. 
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Threats to External Validity . External validity concerns the generalizability of findings 
across times, settings, and persons. For example, a retention researcher completes research 
showing that a mentoring program implemented on ten Afro-American female freshman students 
at a small, midwestern liberal arts college was effective in improving the next year's sophomore 
retention rate for these students. One would have to be careful in generalizing these findings 
across different types of students in different types of institutions (e.g., to North Carolina A&T or 
to a predominantly white university). Examples of this nature are threats to external validity of 
the findings and recommendations. 

An Example of a Retentioa-Focused Program Evaluation 

A host of factors have been studied in relation to student adjustment and effectiveness, 
including skill deficits, values/attitudes, expectations, and institutional attributes. Many student 
support initiatives have focused on influencing these variables. The literature prov : des guidelines 
and suggestions for evaluating such interventions and a range of evaluation examples. Arbitrarily, 
we will describe the evaluation of one intervention, a bridge program, as a recent example of an 
evaluation effort which is retention-oriented. 

The Bridge Program Concept 

The general orientation model entails a three-phased effort: pre-admissions (usually 
written communication), pre-enrollment (e.g. summer programs for incoming freshmen) and initial 
enrollment (i.e., just before classes begin and throughout the first term). Traditionally, summer 
programs or pre-enrollment interventions involve placement testing, academic advising, 
registration, and orientation to the campus and student organizations (Perigo & Upcraft, 1989). 
Single- and multiple-day programs prevail across the nation. The larger the institution, the more 
likely the presence of these orientation innovations, reports the National Orientation Directors 
Association. 
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Efforts to provide a simulation of the college experience in the pre-enrollment phase 
brings us to the bridge program concept illustrated here. Many universities have tried their 
particular version of a realistic preview of the college experience. Our example of the preliminary 
evaluation of a summer intervention to assist minority students emanates from the Georgia 
Institute of Technology. 

CHALLENGE is a summer bridge program started by the Office of Minority Educational 
Development (OMED) to assist minority student adjustment to college life. This intervention 
targets all minority students that have been accepted at the institution. The students are invited to 
participate in a 5-week session during the summer. All who decide to attend are required to pay a 
fee for books and activities, as well as provide their own transportation to and from Georgia 
Tech. Throughout the program, students live in residence halls, take classes (including chemistry, 
calculus, and a "college survival" course), and participate in various extra-curricular group 
activities. 

In the spring of 1992, the senior author and three graduate students were asked by the 
President's Office to develop a questionnaire that would help evaluate the effectiveness of the 
CHALLENGE program. The survey development and implementation consisted of six stages: 
(1) literature review; (2) definition of objectives; (3) selection of variables for measurement; (4) 
pretests and administration of the questionnaire; (5) interpretation of results; and (6) 
recommendations and implications. 

Stage 1: Literature Review . In this stage, the researchers attempted to identify the key 
variables for program evaluation. The focus of the review was on M non-cognitive M factors that 
may affect student retention (i.e. students' perceptions, opinions and attitudes about their social 
and academic environment.) The reasoning was that variables found to be positively linked to 
retention might be influenced by the CHALLENGE program. Numerous variables related to 
student retention were identified. 
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Stage 2: Definition of Program Objectives. Since the criteria for evaluation depend on 
stated goals, researchers found it necessary to learn the objectives of the program as they related 
to minority student retention. At this time, there were no official objectives for the program. 
Thus, the OMED mission statement was used, with program directors providing further 
clarification of the stated goals. This goal identification process both ensured that the 
questionnaire would adequately reflect the objectives of the program and helped reduce the large 
number of potentially relevant variables to a more manageable set. It was determined that the 
goals of CHALLENGE were to (1) provide incoming students with the information and tools 
needed to effectively plan ahead and be "good gamblers" in dividing their efforts, (2) help 
participants develop self-discipline and (3) give students a realistic preview of freshman-level 
classes and some instruction in needed skills and strategies. 

Stage 3: Selecting Variables for Measurement . Twenty CHALLENGE participants were 
interviewed to determine why students chose to attend the program. The intentions and 
expectations identified would be incorporated into the survey instrument to evaluate the program. 
The instrument development sample of participants reported that meeting people like themselves, 
"fitting in," and the opportunity to "prepare for the fall," to be reasons for attending the bridge 
program offered by Georgia Tech. Ultimately, the variables used in the survey instrument were 
chosen because they reflected the literature on retention, the mission of CHALLENGE, and the 
expectations of the pilot sample. It was determined that the following factors would be used to 
assess program effectiveness: 1) perceived academic and social fit, 2) use of coping strategies, 3) 
level of anxiety, 4) degree of preparedness, and 5) self-efficacy, which is defined as the students 
subjective assessment of his/her abilities in general and their degree of college readiness. The 
survey instrument will be available for review and adaptation by our colleagues. 

Stage 4: Pretests and Administration of the Questionnaire. After a pretest tryout, the 
questionnaire was administered to pre-enrollees attending the summer program. The survey 
included measurement scales of the selected variables and open-ended questions to collect 
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information regarding students 1 reasons for participating in CHALLENGE, their impressions of 
Georgia Tech, as well as the participants background and an estimate of educational commitment. 
Forty-five students completed the first administration of the survey during the orientation session. 
At the conclusion of the program, forty of the original participants provided responses to the 
same questionnaire. In addition, students were asked to provide recommendations for 
programmatic changes and to describe positive and negative events that occurred during this 
abbreviated college experience. 

Stage 5: Interpreting the Results . Once all the surveys had been collected from the second' 1 
administration, the information provided was analyzed with the help of a computer. In order to 
determine whether a change had occurred, the responses given during the orientation session were 
compared to those given during the wrap up session. Items measuring the same variable 
construct were added to form a composite score for that variable. The composite score for that 
variable for the first questionnaire was compared to the composite score computed from 
responses for the second administration of the questionnaire. The analysis of variance statistical 
test was used to determine whether significant changes in scores had occurred for each variable. 

The results indicated that the CHALLENGE program affected perceived social fit, 
coping, and preparedness of the participants. After the program, students reportedly felt an 
increased sense of fitting in the social environment at Georgia Tech and felt that they were more 
capable of organizing and managing their time. They also believed that they were more 
knowledgeable about locations on campus, resources available to them, and ways to spend their 
free time than when they first entered the program. In addition, a trend toward a reduction of 
anxiety was seen in participants completing CHALLENGE. However, changes in perceived 
academic fit and self-efficacy were not evident during the pre-enrollment session on campus. 

Stage 6: Recommendations and Implications . Evaluation results generally supported the 
belief that CHALLENGE served to benefit incoming freshmen minority students. Most 
participants cited getting a "head start" on the college experience as a major reason for deciding 



!3 



Evaluation Considerations 

13 

to attend the summer bridge program. The fact that students felt better prepared for the fall and 
adopted strategies to become effective students fulfilled the goals set by the program's directors. 

Although the available results are encouraging in that the program lived up to the 
expectation of helping students prepare for and adjust to the college experience, the evidence did 
not document CHALLENGE as an effective retention intervention. Due to the longitudinal 
nature of process evaluation, Schoob, Bollar and Collins (1992) provided several 
recommendations for follow-up data collection and analysis. The incorporation of the suggestions 
into ongoing evaluation of CHALLENGE would better demonstrate a link between participation 
in the CHALLENGE program and increased retention of minority students and better establish 
the effectiveness of CHALLENGE by addressing one or more of the threats to validity described 
above. 

1) Correlate the post-test composite scores of variables influenced by CHALLENGE with 
first quarter grades to see if these scores predict grades. 

2) Compare first quarter grades of CHALLENGE participants to a similar sample of 
minority students not participating in CHALLENGE as well as a group of non-minority students 
to determine whether attending the program leads to better academic performance. 

3) Collect information regarding how realistic CHALLENGE participants' expectations 
were about Georgia Tech and compare them to the expectations of a non-CHALLENGE minority 
group and a non-minority group. 

4) Use a measure of realism to determine whether realistic expectations are related to 
outcomes such as academic and social satisfaction with Georgia Tech, academic performance, and 
retention. 

5) Continue to track the CHALLENGE participants in order to see whether participation 
in CHALLENGE, or specific factor scores representing survey responses, are predictive of 
retention. 
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This example of a planned evaluation illustrates the growing awareness that meaningful 
information, and data, can be obtained for ongoing student interventions. Although various 
constraints precluded the collection of "evidence" in all four criterion categories mentioned early 
in this paper (i.e., reaction, learning, behavioral, and results criteria), the student reaction data 
enabled subsequent program adaptation and the report became a sales tool for funding 
continuation. The evaluation represents an initial attempt at uncovering the key components of an 
effective bridge program. In addition, the process of evaluation was instrumental in stimulating 
decision makers to begin thinking of evaluation in continuous improvement terms and providing 
input which allows more focused goal setting. 

Promotion of Project Evaluation 
Encouragement is in order for all of us in higher education in terms of the promise offered 
by current evaluation concepts and technology. The lesson for SUCCEED participants, and 
others, is to reach into the diverse literature on student retention (and attrition) and to plan 
appropriately for a systematic evaluation of our ongoing interventions in engineering 
improvement. The methodology is available. The link between outcome variables, properly 
conceived, and the numerous "causes" can be confirmed through our shared efforts. 

The multivariate model testing underway is admirable, providing a better understanding of 
the student persistence phenomenon (e.g., see Cabrera, Nora, & Castaneda, 1993; Gillespie & 
Noble, 1992; Pascarella & Terenzini, 1983; Stage, 1988). Fortunately, data from college records 
and student surveys used in sophisticated models of this nature are beginning to provide causative 
inferences relating to the academic and social dimensions of undergraduate retention. However, 
caution is issued not to generalize too freely from campus-specific studies. 

Our literature reviews are uncovering a host of descriptive studies which address some 
component of the general student retention "system." Despite a plethora of methodological flaws, 
the longer view is positive due to the commitment of so many educators and governmental bodies 
engaged in the continuos improvement of higher education. 
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Under the aegis of the talent development approach (Astin, 1991), renewed attention is 
directed toward interventions as well as curriculum revision. In particular, the first year of college 
is getting increased emphasis based on the alarming attrition figures (Upcraft & Gardner, 1990). 

As in the aforementioned example of one bridge program, evaluation can be accomplished 
-and should be. We should not fall prey to the conceptualization of a project ai d the means for its 
implementation without adequate recognition of the need to assess outcomes relevant to the 
stated objectives. Within almost any project context relating to the SUCCEED mission to 
improve engineering education, we have the evaluation technology among interested and able 
human resources employed by the eight SUCCEED institutions. 



Evaluation Considerations 

16 

References 

Astin, A.W. (1975). Preventing students from dropp in g out . San Francisco: Jossey-Bass. 
Astin, A.W. (1991). Assessment for excellence . New York: American Council on Education, 

MacMillan Publishing Co. 
Astin, A.W. & Panos, R.J. (1969). The educational and vocational development of college 

students . Washington, D.C.: American Council of Education. 
Boyer, E.L. (1987). College: The undergraduate experience in America . Report of the Carnegie 

Foundation for the Advancement of Teaching. New York: Harper & Row. 
Cabrera, A.F., Nora, A., & Castaneda, M.B. (1993). College persistence: Structural equations 

modeling test of an integrated model of student retention. Journal of Higher Education, 

64, 123-139. 

Campbell, D.T., & Stanley, J.C. (1966). Experimental and quasi-experimental design for 

research . Chicago: Rand McNally. 
Cook, T.D., Campbell, D.T., & Peracchio, L. (1990). Quasi experimentation. In M.D. 

Dunnette & Hough, L,M. (Eds.), Handbook of industrial and organizational psychology 

(pp. 491-576). Palo Alto, CA: Consulting Psychologist Press. 
Gillespie, M, & Noble, J. (1992). Factors affecting student persistence: A longitudinal study. 

ACT Research Report Series. 92-4 . 
Kirkpatrick, D.L. (1976). Evaluation of training. In R.L. Craig (Ed.), Training and development 

handbook . New York: McGraw Hill. 
Kulik, C.C., Kulik, J. A., & Shwalb, B.J. (1983). College programs for high risk and 

disadvantaged students: A meta-analysis of findings. Review of Educational Research, 

53,397-414. 

Lavin, D.E. (1965). The prediction of academic performance: A theoretical analysis and review 
of research . New York: Russell Sage Foundation. 



!7 



Evaluation Considerations 

17 

Levin, J.R., & Levin, ME. (1993). Methodological problems in research on academic 

retention programs for at-risk minority college students. Journal of College Student 
Development. 34 . 118-124. 
National Institute of Education (1984). Involvement in learning: Realizing the potential of 

American higher education . Washington, D.C.: U.S. Department of Education. 
Pascarella, E,T., & Terenzini, P.T. (1983). Predicting voluntary freshman year 

persistence/withdrawal behavior in a residential university: A path analytic validation of 
Tinto's model. Journal of Educational Psychology. 75. 215-226. 
Perigo, D.J., & Upcraft, M.L. (1990). Orientation programs. In ML. Upcraft & J.N. Gardner 

(Eds.), The freshman year experience . San Francisco: Jossey-Bass. 
Schoob, C, Bollar, S., & Collins, W. (1992). CHALLENGE program. Development. 

administration, and analysis of a survey for assessing effectiveness . Technical Report, 
Atlanta, GA: Georgia Institute of Technology. 
Schwarz, P. A. (1980). Program devaluation: Can the experiment reform? InE.R Loveland 

(Ed.), Measuring the hard-to-measuie . San Francisco: Jossey-Bass. 
Stage, F.K. (1988). University attrition: LISREL wiin logical regression for the persistence 

criterion. Research in Higher Education. 20. 343-357. 
Tinto, V. (1975). Drop-out from higher education: A theoretical perspective on recent research. 

Review of Educational Research, 45 . 89-125. 
Upcraft, M.L., & Gardner, J.N. (1990). The freshman year experience . San Francisco: Jossey- 
Bass. 

Wortman, P.M. (1983). Evaluation research: A methodological perspective. Annual Review of 
Psychology. 34 . 223-260. 



'8 



